fix(docker): expand ~ and $VAR in extra_mounts destinations - #128

The preview experiment shipped its login mount commented out with a
`$HOME/.uipath` destination, blocked on destination-side `$VAR` expansion
(UiPath/coder_eval#128). It doesn't need it: nightly.yaml and smoke.yaml
already authenticate the in-container `uip` from `~/.uipath:/.uipath:rw`,
a literal destination that validates on the pinned coder_eval 0.10.2.

Adopt that mount verbatim rather than deriving a "more correct" one. It is
the arrangement with a 500-task/night track record, and a login the CLI
cannot see fails tasks on their tenant calls, which scores as a capability
problem rather than a config error. flow-v2-preflight.sh mounts the same
destination, so its "uip reports a live login" check is now the empirical
test of the unified path.

same-ground-headtohead.yaml gets the same treatment, dropping both
`/home/tmatup/...` hardcodes and the comment deferring them to
UiPath/coder_eval#100.

One `$VAR` destination survives, on the repo-root mount, and it is
unrelated to auth: DockerDriverConfig can forward host env vars but cannot
set container ones, so criteria see `$SKILLS_REPO_PATH` with its forwarded
host value and the mount has to land there. That single line is what still
wants coder_eval#128.

Renames make the Flow v2 scope legible:
  tests/experiments/preview-maestro-sdk.yaml -> flow-v2-preview.yaml
  tests/docker/preflight.sh                  -> flow-v2-preflight.sh

same-ground-headtohead.yaml keeps its name: per tests/README.md it is a
general campaign comparison arm (pinned skills-image:sg1, SG_EMPTY_SKILLS
blanking installed skills), not the v1 half of a Flow v2 head-to-head.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

UiPath/coder_eval#128 shipped in 0.11.2, so `_validate_extra_mount` now
expands `~` and `$VAR` in a destination and the flow-v2-preview repo-root
mount loads instead of aborting.

Carries 0.11.0's breaking changes into the whole suite: the cap-drop
anti-cheat window on every `docker run`, and directory-only
`task.reference`. The ~298 `reference: {file:}` tasks under
uipath-troubleshoot are skipped with a migration error until #2707 lands
their migration.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…dk-docker

Keeps this branch's 0.11.2 pin over main's 0.11.1: #2707 moved the pin far
enough to carry its own directory-only `task.reference` migration, and 0.11.2
adds UiPath/coder_eval#128 on top, which the flow-v2-preview repo-root mount
needs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…#2728)

* feat(tests): runnable docker setup for the preview Maestro SDK skills

Running the three preview/uipath-maestro-{flow,case,bpmn} builder-SDK skills
as the only skill catalog needed four non-obvious pieces of setup, none of
them recorded anywhere. Each missing piece scores as a capability failure
rather than a config error, so the results look plausible and are not.

Dockerfile: add a runtime npmrc for the @UiPath scope plus
NPM_CONFIG_USERCONFIG. The build-time npmrc is written to globalconfig and
deleted in the same RUN because it carries the literal token, so the image
ships no npm auth at all; this layer adds a token-LESS npmrc referencing
${NODE_AUTH_TOKEN}, which npm expands at read time. NPM_CONFIG_USERCONFIG is
what makes it reachable: npm resolves userconfig to $HOME/.npmrc, and the
docker runner forwards --env HOME with the HOST value on purpose, overriding
the image's /root, so npm looks in a directory the container does not have,
never maps @UiPath to GitHub Packages, and 404s on the public registry.
Without this, `npm install @uipath/flow-sdk` fails and every builder-SDK
compile fails with it.

tests/experiments/preview-maestro-sdk.yaml: the catalog is narrowed by
pointing agent.plugins[].path at preview/, which shadows the same-named v1
skills. That path does double duty as a bind mount, so narrowing it also
drops the repo-root mount that nightly gets for free, and criteria across the
flow and case suites that shell out to tests/tasks/**/_shared/*.py then exit
2. extra_mounts restores the repo root explicitly. File-based auth is
documented as a commented block: its destination has to equal the host $HOME
for the same forwarded-HOME reason, and mounting to /root/.uipath instead
yields "Not logged in" on every tenant call.

tests/docker/preflight.sh: one container, run with the host HOME forwarded
exactly as the harness does, asserting the eleven preconditions a full run
depends on -- login state readable, SDK installs, and check/compile/check plus
the product scaffold all succeeding. A manual `docker run` without --env HOME
authenticates against the image's /root and reproduces none of these failures,
which is why they are easy to miss.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor(tests): unify auth on the nightly mount, rename to flow-v2-*

The preview experiment shipped its login mount commented out with a
`$HOME/.uipath` destination, blocked on destination-side `$VAR` expansion
(UiPath/coder_eval#128). It doesn't need it: nightly.yaml and smoke.yaml
already authenticate the in-container `uip` from `~/.uipath:/.uipath:rw`,
a literal destination that validates on the pinned coder_eval 0.10.2.

Adopt that mount verbatim rather than deriving a "more correct" one. It is
the arrangement with a 500-task/night track record, and a login the CLI
cannot see fails tasks on their tenant calls, which scores as a capability
problem rather than a config error. flow-v2-preflight.sh mounts the same
destination, so its "uip reports a live login" check is now the empirical
test of the unified path.

same-ground-headtohead.yaml gets the same treatment, dropping both
`/home/tmatup/...` hardcodes and the comment deferring them to
UiPath/coder_eval#100.

One `$VAR` destination survives, on the repo-root mount, and it is
unrelated to auth: DockerDriverConfig can forward host env vars but cannot
set container ones, so criteria see `$SKILLS_REPO_PATH` with its forwarded
host value and the mount has to land there. That single line is what still
wants coder_eval#128.

Renames make the Flow v2 scope legible:
  tests/experiments/preview-maestro-sdk.yaml -> flow-v2-preview.yaml
  tests/docker/preflight.sh                  -> flow-v2-preflight.sh

same-ground-headtohead.yaml keeps its name: per tests/README.md it is a
general campaign comparison arm (pinned skills-image:sg1, SG_EMPTY_SKILLS
blanking installed skills), not the v1 half of a Flow v2 head-to-head.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(tests): trim flow-v2 comments and drop product checks from preflight

Comment volume was out of proportion to the config it explains, in the
experiment, same-ground, the Dockerfile npmrc layer and the README.

flow-v2-preflight.sh loses its six product-behaviour checks (flow check
--source / compile / emitted / check --compiled / solution init / scaffold)
and the inline TS flow they needed. Those are what the eval measures, and
pinning CLI verbs in a gate script only rots. What remains is the six
config assertions whose failures score as capability problems instead:
forwarded HOME, npm userconfig, @UiPath registry, token reachability, a
live uip login, and an in-sandbox flow-sdk install. 82 lines to 52.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor(tests): drop flow-v2-preflight.sh for a one-line login check

The script's only load-bearing assertion was that the `/.uipath` login mount
resolves under the forwarded host HOME. That is one `docker run`, now in
tests/README.md. The rest either duplicated what a failing run already
reports or pinned CLI verbs that drift.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor(tests): move preview npm auth out of the shared image into pre_run

The npmrc layer set ENV NPM_CONFIG_USERCONFIG on tests/docker/Dockerfile,
which changes npm resolution for every suite, not just the Flow v2 preview.
Several skills tell agents to `npm install -g @uipath/cli`, which is
public-npm-only for stable releases and carries no registry override, so a
global @UiPath -> GitHub Packages mapping is a hazard the preview run has no
business creating.

Revert the Dockerfile and write the npmrc from the experiment's pre_run
instead, into $HOME, which is where npm resolves userconfig, so it holds
whatever HOME the runner forwards and needs no NPM_CONFIG_USERCONFIG at all.
pre_run executes inside the sandbox before the agent and fails the task
loudly on error, so a broken write cannot masquerade as a capability miss.
${NODE_AUTH_TOKEN} stays literal; npm expands it at read time.

Only flow-v2-preview.yaml needs this. No other experiment installs an
@UiPath package in-sandbox.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* ci(rpa-smoke): cap anthropic below 1.0 for the pinned coder-eval

anthropic 1.0.0 (2026-08-20) moved its HTTP layer to httpx2. coder-eval
0.10.2 imports httpx in judge_bedrock without declaring it, so it only
ever resolved transitively through anthropic 0.x. On a fresh install
today llm_judge fails to import and every task carrying an llm_judge
criterion errors at setup with score 0.00.

Reproduced against the released wheel, nothing branch-specific:

  uv pip install "coder-eval==0.10.2"
  -> anthropic 1.0.0, no httpx
  -> import coder_eval.criteria.llm_judge
     ModuleNotFoundError: No module named 'httpx'

  uv pip install "coder-eval==0.10.2" "anthropic<1.0"
  -> anthropic 0.125.0, httpx 0.28.1, llm_judge imports

coder_eval main already switched to httpx2, so this cap comes off with
the next .coder-eval-version bump.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* revert: "ci(rpa-smoke): cap anthropic below 1.0 for the pinned coder-eval"

This reverts commit 93f2850.

The forward fix is already open as #2707, which moves the pin to 0.11.1 —
the coder_eval release carrying beeceddc, "bump anthropic to 1.0.0,
migrate Bedrock judge path to httpx2" — and exempts anthropic from the
safe-chain package-age gate in the same four workflows.

Keeping the cap would break that merge rather than help it: 0.11.1
declares anthropic>=1.0.0,<2.0.0, so the two constraints are
unsatisfiable and the install step would fail outright.

  uv pip install "coder-eval==0.11.1" "anthropic<1.0"
  -> No solution found when resolving dependencies

The cap also would not have turned the check green. With httpx restored
the tasks reached the agent and then hit a second, unrelated Windows
break: claude-agent-sdk 0.2.144 cannot find a native claude.exe, so all
three crash with agent_crash after 3 attempts.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore(tests): bump the coder-eval pin to 0.11.2

UiPath/coder_eval#128 shipped in 0.11.2, so `_validate_extra_mount` now
expands `~` and `$VAR` in a destination and the flow-v2-preview repo-root
mount loads instead of aborting.

Carries 0.11.0's breaking changes into the whole suite: the cap-drop
anti-cheat window on every `docker run`, and directory-only
`task.reference`. The ~298 `reference: {file:}` tasks under
uipath-troubleshoot are skipped with a migration error until #2707 lands
their migration.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* ci(smoke): exclude claude-agent-sdk 0.2.144 on the Windows RPA runner

0.2.144 is the first release in months published without a win_amd64
wheel, so uv falls back to the sdist and nothing bundles claude.exe. All
three RPA smoke tasks crash at agent_crash before doing any work.

coder-eval requires claude-agent-sdk>=0.2.124 with no upper bound, so
exclude that single release rather than capping the range: 0.2.143 has
the wheel, and a later release that restores it resolves normally.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>